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PREFACE 


This  report  documents  the  work  conducted  under  the  Office  of  Naval  Research 
contract  No.  N00014-77-C-0401  between  May  15,  1977  and  February  14,  1978. 
This  work  was  performed  by  the  Logistics  Support  and  Services  Division  of 
the  Boeing  Aerospace  Company  and  the  Tactical  and  Training  Systems  Division 
of  Logicon,  Inc.,  a subcontractor  to  Boeing  for  this  program.  The 
primary  results  of  the  Boeing/Logicon  effort  are  contained  in  the  Integrated 
Applications  of  Automated  Speech  Technology  Program  Plan. 

The  authors  are  indebted  to  the  following  persons  for  their  guidance  and 
contributions: 

CDR  Donald  C.  Hanson,  Office  of  Naval  Research,  Technical  Contract 

Monitor 

CDR  Patrick  M.  Curran,  Naval  Air  Development  Center 

Within  the  Boeing  Aerospace  Company,  the  program  was  directed  by  Mr.  W.  J. 
Hebenstreit  of  the  Crew  Systems  Organization.  The  Logicon  effort  was  di- 
rected by  Mr.  L.  D.  Egan.  Other  Logicon  personnel  whose  consultation  dur- 
ing the  program  contributed  significantly  to  the  work  described  herein  in- 
clude: Mr.  M.  W.  Grady  and  Mr.  C.  R.Durfee. 
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1.0  INTRODUCTION 


1.1  Purpose 

The  purpose  of  this  technical  report  is  to  document  the  work  performed  and 
the  results  obtained  during  the  period  of  this  contract.  It  is  intended 
that  this  report  be  of  considerable  assistance  to  the  users  of  the  Integrated 
Applications  of  Automated  Speech  Technology  Program  Plan.  It  will  assist 
them  in  their  understanding  of  the  methodology  and  limitations  of  both  this 
study  program  and  the  aforementioned  program  plan  for  recommended  test  and 
evaluation  of  various  automated  speech  technology  (AST)  applications/projects. 

The  objective  of  the  study  program.  The  Integrated  Applications  of  Automated 

l 

Speech  Technology,  was  to  develop  a methodology  for  integrated  applications 
• of  automated  speech  technology  to  both  Navy  operational  and  training  situations. 

1.2  Scope 

\ The  use  of  the  term  automated  (or  automatic)  speech  technology  (AST)  refers 

to  both  automatic  speech  recognition  (ASR)  and  speech  generation.  This  study 
program  covers  both  aspects  of  AST.  However,  because  ASR  is  the  more  diffi- 
cult aspect  of  AST  to  accomplish,  it  has  received  more  emphasis  in  this  study 
program. 

The  study  program  was  divided  into  essentially  five  tasks  in  order  to  accom- 
plish the  above  indicated  program  objective.  The  first  task  was  to  establish 
a baseline  of  present  day  AST  from  which  to  proceed.  The  second  through 
! fourth  tasks  were  to  analyze  and  define  appropriate  tasks  for  application 

! of  AST  to  each  of  the  three  areas:  a)  crew  station  design,  b)  performance 

measurement,  and  c)  training  functions.  The  fifth  and  final  task  was  to  de- 
velop a long  ranae  test  and  development  program  plan  for  validation  of  the 
above  listed  applications  concents.  The  first  four  of  these  tasks  are  described 
in  detail  in  Section  2.0  of  this  report.  The  scheduled  accomplishment  of 
i these  tasks  in  relation  to  the  major  program  milestones  is  shown  in  the 

following  section. 

i 

i 


4 


1.3  Program  Schedule 


r 
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The  major  portions  of  the  program,  including  the  previously  described  five 
tasks  were  accomplished  over  a seven  month  period  beginning  on  May  15,  1977. 
This  seven  month  effort  was  followed  by  a two  month  period  for  customer 
review  of  the  program  documentation  and  refinement  as  necessary.  The  sub- 
mittal of  the  approved  program  plan  and  the  final  report  was  made  on  February 

14,  1978.  Figure  1.3-1  shows  the  program  schedule  of  tasks  and  significant 
events  as  they  actually  occurred.  Although  an  informal  initial  meeting  was 
held  in  Orlando,  Florida  on  May  3,  1977,  to  discuss  the  program,  the  official 
kickoff  meeting  occurred  on  May  26  after  the  contract  start  date.  The  meeting 
was  held  at  Logicon,  Inc.,  in  San  Diego,  California  The  purpose  of  the  meet- 
ing was  to  review  the  proposed  program  schedule  and  to  agree  on  the  details 
of  the  tasks  to  be  accomplished  and  the  type  of  mission  to  be  used  as  a base- 
line for  such  tasks.  A significant  additional  topic  of  discussion  at  the 

kickoff  meeting  was  the  July  5-6,  1977  Apportionment  Review  and  the  need  to 
present  preliminary  results  from  this  program  at  that  meeting.  Following  the 
kickoff  meeting,  and  prior  to  ti.e ‘mid-term  review,  several  informal  meetings 
occurred  between  various  Navy,  Boeing,  and  Logicon  personnel. 

The  Mid-Term  Review  took  place  at  the  Office  pf  Naval  Research  in  Arlington, 
Virginia  on  October  4,  1977.  Program  progress  and  plans  for  completion  were 
presented  by  Boeing  and  Logicon.  Outlines  for  the  Program  Plan  and  the  Final 
Report  were  presented  for  review  and  coiirients. 

The  initiation  of  work  on  the  five  major  program  tasks  occurred  in  a slightly 
different  order  than  originally  planned.  As  can  be  seen  from  the  Figure  1.3-1 
schedule,  the  Technology  Review  Task  did  not  begin  until  approximately  a month 
and  a half  after  the  start  of  the  program.  The  reason  for  the  delay  in  Task  1 
was  to  concentrate  the  program  efforts  on  Task  2,  Crew  Station  Design.  This 
task  was  started  earlier  than  planned  in  order  to  provide  preliminary  data  in 
support  of  AST  and  this  program  for  the  aforementioned  Apportionment  Review 
meeting.  The  accomplishment  of  the  remaining  three  tasks  occurred  in  the 
order  originally  planned. 


5 


PROGRAM  SCHEDULE 


, Program  Schedule 


In  addition  to  the  minutes  of  the  Kickoff  and  Mid-Term  Review  meetings, 
the  documentation  for  this  program  has  consisted  of  four  bi-monthly  pro- 
gress reports,  preliminary  drafts  of  both  a program  plan  and  a final 
report,  and  approved  final  versions  of  these  same  two  documents. 


i 
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2.0  TASKS  ACCOMPLISHED 

The  following  four  sections  describe  the  details  of  the  work  accomplished 
during  this  study  program.  Section  2.1  contains  the  complete  results  of  the 
current  technology  review.  Section  2.2  covers  the  task  work  performed  in  the 
analysis  of  AST  application  to  crew  station  design.  The  additional  appli- 
cations of  AST  to  both  operator  performance  measurement  and  training  are  con- 
tained in  Section  2.3.  Section  2.4  describes  the  process  used  to  compare 
each  of  the  possible  AST  applications  taken  from  crew  station  design,  per- 
formance measurement  and  training.  These  applications  are  compared  on  the 
basis  of  feasibility,  risk,  cost,  and  impact  in  order  to  choose  the  most 
promising  projects  for  inclusion  in  the  program  plan. 

2.1  AST  Technical  Review 

The  following  material  was  prepared  as  Task  1 of  the  study  program.  This 
task  was  performed  and  the  results  included  herein  in  order  to  familiarize  the 
users  of  the  report  and  the  program  plan  with  the  state-of-the-art  in  AST. 

These  applications  and  their  supporting  data  may  be  used  as  AST  baseline  in- 
formation from  which  Navy  managers  can  subsequently  plan  AST  applications. 
Additional  survey  information  as  to  the  speech  understanding  field  are  now 
being  conducted  by  Speech  Communications  Research  Laboratory,  Inc.  However, 
these  data  will  not  be  available  for  some  time.  Several  presentations  and  pub- 
lications, including  two  forthcoming  books  will  report  such  work. 

In  this  decade,  the  technology  of  AST  and  automatic  speech  recognition  (ASR) 
in  particular,  has  progressed  from  the  status  of  an  R&D  topic  to  that  of  a 
viable  alternative  to  conventional  man/machine  interface  devices.  While  ASR 
progress  has  not  met  the  goals  established  by  Newell,  et.  al . , (1971),  for  the 
ARPA  Speech  Understanding  Research  Program,  it  has  encouraged  application  of 
ASR  to  various  systems  where  vocabulary  requirements  are  consistent  with  the 
capability  of  commercial ly-available,  isolated  word  devices.  As  described  by 
Beek,  Newberg,  and  Hodge  (197A  these  applications  serve  government  systems 
of  various  types.  These  include: 

a.  Training  systems  (and  performance  measurement). 

b.  Crew  station  systems  on  aircraft. 

c.  Command  verification  systems  on  ships. 

d.  Intelligence  data  handling. 
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In  several  of  these  applications,  development  has  progressed  to  the  point 
where  demonstrations,  experiments,  and  tests  have  been  conducted  on  the 
system  or  subsystem  level  to  verify  the  feasibility  of  AST  as  a data  inter- 
face in  man/machine  systems. 

2.1.1  Rationale  Behind  ASR  Applications 

To  understand  the  impetus  for  ASR  application,  the  human  roles  in  large  systems 
as  they  are  presently  configured  must  be  examined  as  well  as  the  assistance 
speech  technology  might  offer  to  these  roles.  With  the  advent  of  modern, 
highly-sophisticated  systems,  the  scope  of  man's  involvement  has  burgeoned 
to  a point  where  his  performance  shows  signs  of  degradation  as  a result  of 
work  overload.  Cockpit  design,  for  example,  has  not  kept  pace  with  overall 
weapon  system  design.  Dated,  slow  input  devices  still  are  being  employed 
to  enter  data  into  the  system.  This  requires  the  operator  to  share  attention 
between  several  sources  of  information.  Head-up  displays  (HUD)  have  been 
developed  to  deal  with  this  problem  but  serve  only  as  a partial  solution. 

What  is  needed  is  a new  mode  of  inputting  and  outputting  data  into  a system 
that: 

a.  Frees  the  hands  and  eyes  for  other  tasks. 

b.  Does  not  disrupt  attention. 

c.  Is  faster  than  present  modes. 

d.  Is  equally  (or  more)  accurate  than  present  modes. 

Further,  the  new  mode  must  be  compatible  with  environmental  constraints 
and,  above  all,  must  be  acceptable  to  the  operator.  ASR  potentially  offers 
these  advantages.  For  example.  Turn  (1971)  has  indicated  that  speech  is: 

a.  A natural  channel  for  human  connunication. 

b.  Independent  of  the  visual  channel  and  motor  activities. 

c.  Omni-directional  (thus,  no  free  line-of-sight  is  required). 

d.  Informative  about  the  speaker. 

e.  Faster  in  data  rate  (not  necessarily  information  rate)  than  other 
input  modes. 


9 


These  advantages  of  ASR  suggest  that  it  may  be  an  extremely  desirable  input 
mode  if  a device  can  be  constructed  that:  first,  understands  speech  without 
requiring  the  human  to  alter  his  "natural"  speech  habits;  second,  has  suffi- 
ciently high  recognition  accuracy  to  prevent  user  dissatisfaction;  and  third, 
is  cost-effective. 

This  review  will  determine  the  extent  to  which  present  application  programs 
show  promise  for  ASR  as  an  improved  input-output  method. 


2.1.2 


)li cation  Programs 


In  conducting  this  review  it  was  decided  to  concentrate  on  newer  programs  that 
have  applied  automated  speech  technology  (AST)  as  part  of  a larger  system. 

Older  applications,  such  as  baggage  handling  and  inspection  line  systems,  are 
well  documented  elsewhere  and  offer  little  in  the  way  of  state-of-the-art  ad- 
vancement for  AST  application.  Similarly,  research  programs  were  not  reviewed 
except  where  pertinent  to  specific  application  problems.  In  all  of  the  cases 
reviewed,  the  identifcation  and  review  of  present  and  near-term  future  appli- 
cations will  allow  Navy  planners  to  track  applications  that  might  interest  them. 
Thus,  the  purpose  of  the  following  paragraphs  is  to  identify  and  to  summarize 
the  status  of  various  AST  applications  as  potential  contributions  to  subsequent 
Navy  AST  planning  and  programming. 

2. 1.2.1  Operational  Applications  - During  this  AST  review,  a variety  of  pro- 
grams were  examined  which  attempt  to  apply  AST  to  operational  problems  such 
as  cockpit  design,  shipboard  command  verification,  and  paraplegic  support. 

These  applications  have  been  supported  by  several  government  agencies  in- 
cluding DOD,  NASA,  NEW  and  DOT.  The  technical  effort  has  been  supplied  by 
laboratories  within  these  government  agencies  and  by  several  companies,  in- 
cludino  Logicon,  Dialog  Systems,  Analytics,  and  Threshold  Technology.  These 
applications  are  reviewed  by  topic  below. 
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The  earliest  application  of  AST  to  an  operational  military  problem  mas  per- 
formed at  the  Naval  Air  Development  Center  (NADC)  Human  Factors  Laboratory. 

The  program  at  NADC  was  Initiated  to  determine  the  desirability  of  Inter- 
active voice  systems  for  use  In  airborne  weapon  systems  crew  stations.  To 
accomplish  this  effort,  NADC  developed  a voice  recognition  and  synthesis 
(VRAS)  system  which  was  Incorporated  Into  the  centrifuge  at  NAS,  Johnsville, 

Pa.  The  speech  recognition  aspect  of  VRAS  was  developed  using  a voice  command 
system  (VCS)  which  was  developed  by  Scope  Electronics.  The  speech  synthesis 
capability  was  supplied  by  a Votrax  VS-5  speech  synthesis  unit  built  by 
Vocal  Interface.  Development  of  the  system  was  performed  jointly  by  Scope  and 
NADC  personnel,  notably  R.  Wherry,  Jr. 

The  effects  of  flight  on  automatic  speech  recognition  was  determined  by  using 
three  subjects  (two  pilots  and  one  psychologist)  who  were  given  repeated  trials 
in  the  VRAS-equipped  centrifuge  to  determine  the  effects  of  vibration,  G,  O2 
mask,  mission  duration,  and  cockpit  temperature  on  voice  quality.  The  results 
showed  that: 

a.  Voice  quality  degrades  after  0.5  hours  with  an  0^  mask. 

b.  Voice  quality  degrades  under  high  (±  3G)  vibration. 

c.  Voice  quality  degrades  under  higher  levels  of  G- 

These  results  were  obtained  with  a baseline  of  80  percent  recognition  accuracy 
with  the  VCS. 

The  next  phase  of  the  NADC  program  called  for  improvements  of  the  VCS 
system.  This  was  accomplished  in  two  ways. 


a.  A consistency  bit  was  incorporated  into  the  process  wherein  reference 
patterns  are  established  to  improve  recognition  accuracy.  Improved 
recognition  accuracy  was  noted. 

b.  A syntactic  handler  was  developed  to  facilitate  human  usage  of  the 
isolated  word  VRAS  system  and  simultaneously  assist  In  the  under- 
standing process. 
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The  syntactic  handler  was  tested  with  teletype  input  and  was  operational 
with  100  percent  accuracy  in  real  time.  It  presently  is  being  translated 
into  FORTRAN  IV  (by  Wherry)  under  contract  to  Analytics,  Inc.  Transla- 
tion should  be  complete  within  the  next  few  months  thereby  allowing  its 
use  on  a variety  of  computers  rather  than  solely  on  the  Raytheon  704.  Since 
the  syntactic  handler  only  requires  a recognized  word  or  phrase  as  input, 
it  can  be  used  with  a VIP-100  or  any  other  recognition  device  as  well  as  a 
VCS.  Wherry  estimates  it  would  take  between  one  and  four  weeks  to  develop 
a syntactical  handler  for  another  application. 

The  power  of  the  syntactic  handler  is  that  it  will  allow  the  user  to  vary 
the  syntactical  arrangement  of  words  during  data  entry  without  affecting 
recognition  accuracy.  Thus,  the  natural  quality  of  speech  as  a data  entry 
means  is  preserved. 


An  informal  attempt  to  incorporate  VRAS  into  the  LAMPS  simulator  at  NAS,  | 

I 

Johnsville  was  made  in  1975  but  interface  problems  between  the  simulator  i 

computer  and  the  Raytheon  704  in  the  VRAS  system  prevented  any  formal 

operability  demonstration.  , 


Crew  Station  Design  - NASA.  A second  application  study  is  being  conducted 
at  the  NASA  Ames  Laboratory  in  California.  At  NASA  Ames,  feasibility  of 
speech  recognition  as  an  input  means  for  a flight  management  system  in 
commercial  aircraft  (including  helicopters)  is  being  investigated.  NASA's 
program  assumes  that  large  digital  computers  will  be  placed  on-board  the 
aircraft;  NASA's  task  is  to  utilize  these  computers  for  maximum  benefit. 

As  a facilitation  of  the  computers,  speech  input  is  thought  to  offer  several 
advantages  to  human  operators,  including  freeing  of  hands  and  some  degree  of 
attention. 

Beginning  in  1972,  NASA  Ames  purchased  a Scope  VCS  recognition  device  (which 
was  the  only  device  commercially  available  then).  Faced  with  poor  recogni- 
tion accuracy,  even  with  a 10  word  vocabulary,  NASA  Ames  opted  to  improve 
recognition  accuracy  before  pursuing  device  application.  By  structuring  their 
vocabulary  and  using  a single  detection  approach  to  recognition,  recognition 
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accuracy  for  a 100  Mord  vocabulary  is  now  98  percent  without  rejection  and 
aln«st  100  percent  with  a 5 percent  rejection  rate.  These  figures  are  suf- 
ficient to  warrant  progression  to  experimentation. 

The  experimental  phase,  which  began  In  January,  1977,  will  compare  speech 
entry  to  keyboard  entry  for  mixed  tracking  tasks.  Speed  and  accuracy  of 
Input  will  be  measured  as  well  as  tracking  accuracy.  Subsequent  experiments 
will  Involve  Identification  of  background  noise,  vibration,  and  motion 
factors. 

After  initial  experimentation,  the  next  step  will  be  to  chain  keyboard  and 
speech  entries  in  a meaningful  navigation/guidance  context  to  compare  per- 
formance on  a visual/manual  tracking  task  in  a high-workload  situation. 

The  overall  goal  of  the  program  is  to  introduce  a voice  operated  system 
into  actual  flight  conditions  for  evaluation  purposes.  With  this  intention, 
discussions  and  demonstrations  with  P-3C  TACCOS  have  been  conducted  with 
favorable  results.  A ruggedized  computer  (Rolm)  also  has  been  purchased 
with  this  application  in  mind. 

2. 1.2. 2 Training  Applications.  As  part  of  this  technological  review,  two 
applications  of  AST  to  training  were  identified.  Both  applications  concerned 
military  training  and  both  were  conducted  by  Logicon,  Inc.  (under  contract 
to  Naval  Training  Equipment  Center,  Orlando,  Florida).  In  addition,  these 
applications  involved  both  speech  synthesis  and  speech  recognition  aspects 
of  AST. 

The  initial  application  of  AST  to  training  was  conducted  as  part  of  the 
Adaptive  Flight  Training  System  (AFTS)  program.  The  following  applica- 
tion concerned  the  use  of  AST  to  train  ground-controlled  approach  (GCA) 
controllers  in  a synthetic  environment.  The  latter  application  Is  called 
CGA  Controller  Training  System  (GCA-CTS).  These  applications  will  be 
discussed  within  the  ensuing  sections. 
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AFTS.  The  initial  application  of  AST  to  training  problems  began  as  a result 
of  a requirement  in  Logicon's  AFTS  program  for  a "synthetic"  GCA  controller 
to  replace  live  "pseudopilots"  in  simulated  GCA  training  of  aircrews.  Speech 
generation  thus  became  the  link  between  the  automatic  controller  and  student 
aircrews  on  simulated  GCA  exercises. 


In  the  1969-1973  era,  a voice  drum  was  the  only  technology  available  for 
computer-controlled  voice  simulation.  This  approach  used  specially  produced 
tapes  on  which  all  words  within  the  vocabulary  were  recorded,  in  short  equal 
length  intervals  (often  one  second  in  length).  The  user's  vocabulary,  such 
as  GCA  phrases,  then  was  constructed  by  combining  several  of  the  recorded 
words  into  one  phrase,  under  computer  control. 


The  result  of  combining  equal  length  words  in  a phrase  was  an  unnatural, 
choppy,  but  intelligible  string  of  sounds  that  were  marginally  adequate  for 
simulation  purposes.  This  approach,  supplied  by  Cognitronics  or  Metrolab, 
was  used  in  the  early  feasibility  demonstration  versions  of  AFTS  with 
reasonable  success. 

By  1974,  several  digital  voice  synthesizers  were  available  on  the  commer- 
cial market.  Logicon  chose  the  Votrax  unit  (made  by  the  Vocal  Interface 
Division  of  the  Federal  Screw  Works)  as  the  replacement  for  the  voice 
drum.  This  new  approach,  which  synthesized  artificial  phoneme-like  sounds 
into  words,  not  only  sounded  more  natural  but  cost  far  less  than  the  voice 
drum  approach. 

The  AFTS,  with  the  improved  Votrax  VS-6  speech  synthesis  unit,  was 
installed  at  Luke  AFB  in  1974  as  an  "add-on"  to  the  F-4E  Weapon  System 
simulator  (A/F  37U-T9).  AFTS*  initial  purpose  was  to  train  aircrews  in 
GCA  phases  of  operational  F-4  missions.  During  1975,  the  scope  of  AFTS 
was  increased  to  include  air-to-air  intercept  and  ground-attack  radar 
phases  of  F-4  simulator  training.  To  add  automation  to  these  additional 
mission  phases,  Logicon  incorporated  automated  speech  recognition  (ASR) 
systems  which  understand  important  tactical  messages  (such  as  those  con- 
tained in  the  operational  brevity  code)  spoken  by  the  aircrew  to  ground 
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controllers.  To  determine  the  feasibility  of  performing  this  function,  a 
Threshold  500  speech  processor  and  Logicon-developed  software  were  installed 
in  AFTS  to  capture  approximately  24  key  messages  (such  as  JUDY,  CONTACT,  etc.) 
spoken  by  the  pilot  and  the  weapon  systems  officer. 

Combined  with  the  already  existing  speech  synthesis  capability,  the  enhanced 
AFTS  provides  an  interactive  system  in  which  aircrew  messages  are  automatically 
understood  and  responded  to  with  the  appropriate  synthesized  voice  outputs. 

The  following  is  an  example  of  the  AFTS-to-aircrew  dialogue: 

AFTS:  "Phanton  1,  cleared  for  reattack" 

Aircrew:  "Say  again" 

AFTS:  "Phantom  1,  cleared  for  reattack" 

Aircrew:  "Roger" 

AFTS:  "Phanton  1,  Contact" 

Aircrew:  "Roger,  contact  is  target" 

Aircrew:  "Phantom  1,  Judy" 

AFTS:  "Roger,  Judy" 

Aircrew:  "Phantom  1,  Lost  Contact" 

AFTS:  "Phantom  1,  you  have  a target  at  258" 

Aircrew:  "Phantom  1,  Roger" 

With  the  vocabulary  of  24  items  shared  between  two  users,  recognition 
accuracy  in  AFTS  was  observed  to  be  lower  than  other  applications  80-90 
percent).  The  lower  accuracy  stems  from  several  problems  that  might  affect 
any  application  outside  a controlled  environment. 

These  problems  are: 

a.  Noise.  A 400  Hz  noise  source  in  the  simulator's  intercom 
system  masks  the  lower  frequency  of  voice  transmission. 
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[ b.  User  acceptance.  Because  the  enhanced  AFTS  has  not  been  fully  ] 

integrated  into  the  simulator  syllabus,  the  aircrews  have  not  | 

been  diligent  in  maintaining  their  voice  file  (or  training  file) 
for  voice  understanding.  i 

c.  Training-using  fidelity.  The  system  does  not  require  the  using 
aircrews  to  "train"  the  speech  recognition  system  in  a mission 
context.  Training  is  done  off-line  by  simply  repeating  a given 
phrase  10  times.  However  necessary,  this  process  has  been  re-  » 

ported  as  "boring"  by  users.  Further,  the  F-4E  syllabus  does  not 
require  crew  members  to  adhere  to  a rigid  vocabulary;  therefore, 
the  training  process  is  frequently  ignored.  This  leads  to  low 
fidelity  between  the  messages  trained  and  used  which,  in  turn, 
effects  recognition  accuracy.  (For  maximal  recognition  accuracy, 
fidelity  should  be  high,  if  not  perfect.) 

It  should  be  noted  that  these  problems  are  the  results  of  the  "feasibility 
study"  status  of  the  enhanced  AFTS.  They  do  demonstrate,  however,  the  neces- 
sity of  dealing  with  device  training  as  a variable  that  can  affect  user  ac- 
ceptance. 

GCA-CTS.  As  a result  of  investigations  into  speech  synthesis  for  AFTS, 

Logicon  became  aware  of  ASR  and  its  potential  for  training  skills  which  are 
primarily  verbal.  One  such  application  is  in  the  air  traffic  control  environ- 
ment where  controllers  are  taught  the  use  of  specific  short  phrases  to  direct 
the  ground  controlled  approach  (GCA)  of  an  aircraft.  Specifically,  ASR  offered 
an  opportunity  to  automate  both  instructor  functions  (e.g.,  performance  mea- 
surement) and  pseudopilot  functions  (e.g.,  simulation  of  pilot's  responses  to 
the  advisories  and  environment)  during  simulated  GCA  practice.  Objective 
measures  of  controller  performance  during  GCA  runs  were  also  provided. 

In  1973,  feasibility  studies  were  conducted  by  Logicon  for  NAVTRAEQUIPCEN 
that  demonstrated  that  sufficient  ASR  technology  existed  to  make  GCA 
controller  training  possible  (Feuge  and  Charles,  1973).  A laboratory  demon- 
stration version  of  a GCA  controller  training  system  (GCA-CTS)  was  developed 
which  used  a vocabulary  of  45  GCA  phrases,  with  each  phrase  composed  of  up  to 
eight  words  whose  length  approached  2-3  seconds.  Operating  in  real-time. 
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the  GCA-CTS  used  a VIP-100  made  by  Threshold  Technology,  Inc.  Recognition 
accuracy  was  observed  to  range  about  95  percent  with  trained  speakers. 

The  first  system  delivery  of  the  GCA-CTS  to  NAVTRAEQUIPCEN  in  1974  repre- 
sented the  initial  application  of  ASR  to  a sophisticated  training  problem. 
Unlike  the  AFTS,  the  entire  automatic  training  concept  (objective  performance 
measurement  and  adaptive  syllabus  control)  is  contingent  on  the  efficacy 
of  the  speech  technology  components  of  the  system. 

Several  advancements  to  the  state-of-the-art  in  ASR  were  developed  for  the 
GCA-CTS  application;  these  included: 

a.  Long  phrases  (2-3  seconds)  which  were  recognized  with  high 
accuracy.  Individual  words  (or  sometimes  2 or  3 word  phrases) 
previously  limited  the  vocabulary. 

% 

b.  A larger  vocabulary  (45  phrases)  which  was  easily  accommodated. 
Previous  applications  used  a maximum  of  32  words. 

c.  Software  which  was  enhanced  to  accommodate  rapid-fire  voicing 
with  less  than  a half-second  between  vocabulary  phrases. 

d.  A digit  extraction  algorithm  which  was  developed  and  used  for 
high  accuracy  recognition  of  the  final  digit  from  a long  phrase. 

e.  Effective  schemes  which  were  developed  for  distinguishing  small 
differences  between  vocabulary  phrases. 

f.  Level  of  confidence  which  was  effectively  used  in  the  recognition 
process  to  distinguish  between  student  and  machine  recognition 
errors. 

g.  The  speech  recognition  subsystem  which  was  packaged  as  a highly 
flexible  set  of  FORTRAN  routines  which  thereby  enabled  the  easy 
modification  of  vocabulary  and  application. 
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Subsequent  developments  of  the  GCA-CTS  incorporated  speech  generation  capa- 
bilities have  provided  improvements  in  the  training  methodology.  Thus,  the 
GCA-CTS  demonstrates  the  integration  of  the  speech  technologies  into  one 
complete  system.  The  limitations  of  the  recognition  technology  (e.g., 
requirements  for  a priori  training  data)  should  present  no  difficulty  in 
GCA-CTS  because  they  are  incorporated  into  the  overall  training  program  by 
having  the  student  learn  the  vocabulary  at  the  same  time  that  the  computer 
is  developing  reference  data.  The  verbal  behavior  of  the  student  is 
critical  to  his  task,  therefore,  he  is  expected  to  be  a willing  and 
cooperative  participant.  The  small  number  of  unnatural  speech  stylizations 
have  been  accepted  readily  and  generally  learned  easily. 

At  present,  the  GCA-CTS  resides  at  the  NAVTRAEQUIPCEN  Human  Factors  Laboratory 
as  a laboratory  demonstration  model  for  further  development.  While  at 
NAVTRAEQUIPCEN,  the  GCA-CTS  has  been  reviewed  by  training  personnel  from 
NATTC  as  well  as  Fleet  controllers.  These  reviews,  which  have  been  largely 
favorable,  have  led  to  further  refinement  by  NAVTRAEQUIPCEN,  particularly  in 
the  training  conceptualization  of  GCA  controller  training.  The  GCA-CTS 
studies  at  NAVTRAEQUIPCEN  (Breaux  and  Goldstein,  1975)  have  shown  that: 

a.  Recognition  accuracy  increases  as  user  experience  with  ASR  increases 
(speech  variability  is  reduced). 

b.  Recognition  accuracy  is  an  inverted  "U"  function  of  controller 
experience  (confidence  in  controlling). 

c.  Recognition  accuracy  varies  as  a function  of  the  type  of  vocabulary 
(some  vocabularies  contain  items  which  are  inherently  similar  to 
one  another). 


These  results  indicate  that  considerable  skill  must  be  used  in  applying  ASR. 
An  approach  must  be  developed  to  reduce  speech  variability  and  account  for 
highly  similar  vocabulary  items  if  recognition  accuracy  (and,  hence,  user 
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Shipboard  Automatic  Cotnnand  and  Response  Verlficatfon  (ACRV).  The  ACRV 
application  study  was  conducted  by  Logicon  for  the  Department  of  Trans- 
portation in  1976.  rts  purpose  was  to  test  the  feasibility  of  a system  where 
orders  from  the  conning  officer  of  a ship  could  be  automatically  monitored, 
compared  to  the  helmsman's  response,  and  verbally  indicated  and  sent  back 
to  the  conning  officer  if  an  error  in  the  helmsman's  response  were  suspected. 
Automation  of  the  command  verification  system  was  achieved  through  the  use  of 
a Threshold  Technology  VIP-100  (for  speech  recognition)  and  a Votrax  VS-6 
speech  synthesis  unit,  which  were  combined  in  a mockup  of  a Coast  Guard 
cutter  bridge  at  Logicon 's  computer  center. 

Demonstrations  of  ACRV  to  DOT  and  Coast  Guard  personnel  have  proven  the 
feasibility  of  ASR  for  conmand  and  response  verification  tasks.  These 
demonstration  tests  indicated  that  speakers  with  little  or  no  prior  experi- 
ence with  AST  averaged  93.3  percent  recognition  accuracy  after  three  hours 
of  practice  with  a vocabulary  of  63  phrases.  Speakers  with  lengthy  AST 
experience  consistently  averaged  over  98  percent  recognition. 

From  the  AST  point  of  view,  the  command  verification  system  offered  several 
design  challenges;  i.e.,  it  used  a large  vocabulary  (64  phrases),  very 
similar  phrases  ("starboard  ahead  1/2"  vs.  "starboard  ahead  2/3"),  and  con- 
siderable speech  stylization  ("indicate. .1 . .1 . .0. . .revolutions").  More- 
over, the  design  of  the  automated  advisories  was  critical  in  the  decision 
of  what,  when,  and  how  the  computer  should  respond  to  a detected  error. 


The  recognition  software  developed  for  the  ACRV  system  was  very  similar 
to  that  of  Logicon's  GCA-CTS.  To  minimize  core  requirements,  all  the 
referenced  patterns  were  stored  on  disk  and  selectively  retrieved  in  real 
time  when  they  were  needed.  Using  this  scheme,  the  vocabulary  size  was 
limited  only  by  practical  considerations  such  as  "training  time"  etc.  The 
ACRV  scheme  would  be  particularly  useful  in  a structured  vocabulary  (such 
as  those  used  by  NASA  Ames)  since  the  amount  of  data  which  must  be  re- 
trieved from  mass  storage  would  be  further  limited. 
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2. 1.2. 3 Hospital  Systems.  Another  type  of  application  is  being  developed 
by  Dialog  Systems,  Inc.  It  seeks  to 'apply  ASR  to  aid  paraplegic  patients 
in  large  hospitals  such  as  the  VA  Hospital  in  Roxbury,  Massachusetts.  The 
hospital  application  will  have  a 105  word  vocabulary,  composed  of  discrete 
words  or  phrases  which  will  be  separated  by  a 0.25  second  pause.  The  system 
is  designed  to  allow  paraplegic  patients  to  perform  the  following  functions 
by  voice: 

a.  Type  on  a typewriter. 

b.  Dial  a telephone. 

c.  Answer  a telephone. 

\ 

d.  Change  TV  chaitnels.  , 

e.  Control  room  lighting.  ,■  i. 

f.  Operate  bed  motors. 

r» 

g.  Operate  a desk  calculator. 

The  vocabulary  is  syntactically  stri/ctu red  much  like  those  on  the  NASA 
Ames  and  VRAS  systems.  Estimated  recognition  accuracy  r’anges  from  96 
percent  to  99+  percent,  depending  upon  speaker  training.  The  system  will 
be  largely  speaker  independent,  using  reference  patterns  derived  from  the 
"Greek  chorus"  of  50  speakers  for  each  specific  vocabulary  item.  For 
individuals  who  observe  low  recognition  accuracy,  a short  training  period 
(consisting  of  six  repetitions  of  key  words)  will  be  used  to  individualize 
the  "chorus"  and  thus  improve  accuracy.- 

The  Dialog  Systems'  "front  end"  uses  32  frequency  channels  and  a comb 

filter  to  produce  /lata  that  are  processed  in  search  of  the  first  two 

formats.  Time  is  normalized.  The  pre-processor  is  composed  of  11  to  14 

boards.  Dialog  Systems  uses  a PDP-11/04  and  232K  of  memory  (MCS)  to 

achieve  a near  real-time  recognition  capability.  Response  time-- is  estimated 

1 

to  be  50  milliseconds  (essentially  real  time)  for  a 100  word  vocabulary. 
Their  system,  at  present,  weighs  600  pounds  and  is  rather^ bulky.  In  sub- 
sequent applications*  Dialog  is  planning  to  reduce  their  system' to  250 
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pounds  and  a voliane  of  from  1.0  to  1.5  cubic  feet.  Cost  ranges  from  $40,000 
to  $60,000  for  a turnkey  system.  Their  voice  output  is  through  a Cogni- 
tronics  31  system. 

Dialog  has  performed  no  user-oriented  studies.  As  their  representative 
indicates,  their  application  (hospital  environment  control)  cannot  be 
accomplished  by  the  patient  independent  of  other  people;  thus,  it  is 
bound  to  be  acceptable. 

2. 1.2. 4 Application-Oriented  Research  and  Development.  In  addition  to  the 
preceding  described  application,  an  application-oriented  research  problem  is 
being  conducted  by  Logicon  for  NAVTUAEQUIPCEN.  The  research  is  aimed  at  the 
development  of  a limited  continuous  speech  recognition  capability  which  could 
be  used  to  understand  (in  real  time)  long  strings  of  digits,  spoken  rapidly 
without  pausing  between  digits. 

Logicon's  approach  is  to  use  a standard  VIP-100  preprocessor,  minicomputer, 
and  special  software.  This  approach  emphasizes  the  latest  trends  in  current 
speech  research,  namely: 

a.  Treating  the  speech  preprocessor  as  a sound  classifier  rather 
than  as  a phoneme  detector. 

b.  Emphasizing  the  derivation  of  the  recognizable  speech  character- 
istics from  real  speech  data, 

c.  Decoding  the  speech  signal  sequentially  rather  than  by  exhaustive 
hypothesis  and  test  methods. 

Continuous  speech,  it  is  postulated,  can  be  characterized  and  recognized 
on  the  basis  of  observing: 

a.  Characteristic  classes  of  output  from  a preprocessor. 

b.  The  order  in  which  these  occur. 

c.  The  characteristic  time  durations  between  the  output  samples. 
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Higher  knowledge  sources,  such  as  semantics,  are  seen  as  irrelevant  to  the 
recognition  process  for  the  chosen  vocabulary. 

If  proven  feasible,  limited  continuous  speech  recognition  would  perhaps  solve 
problems  that  plague  isolated  word  systems,  such  as  recognition  of  key  words 
or  phrases  that  are  imbedded  in  a larger  utterance.  For  example,  recognition 
of  the  specific  heading  in  the  GCA  advisory  "Heading  is  289"  would  be  possible 
without  speaker  stylization  or  vocabulary  stylization  which  is  required  today. 
The  feasibility  of  such  an  approach  to  limited  continuous  speech  recognition 
will  be  tested  late  in  1977. 

In  another  line  of  research  and  development,  an  alternative  speech  recogni- 
tion device  is  being  developed  by  Centigram,  Inc.,  which  will  be  capable  of 
recognizing  a limited  set  of  16  digits  and  other  words  in  real  time.  Centigram' 
device  is  projected  to  be  significantly  less  expensive  than  other  devices  in 
the  commercial  market  (approximately  $3,000)  but  its  accuracy  must  still  be 
proven  before  application  can  be  seriously  considered. 

2.1.3  Human  Factors  Aspects  of  AST 

As  described  by  Turn  (1971),  AST  should  provide  an  exceptional  input/output 
channel  for  humans  because  it  is  natural,  fast,  and  accurate.  However,  little 
empirical  evidence  is  available  to  indicate  how  AST  compares  to  conventional 
input  and  output  devices  concerning  dimensions  such  as  speed,  accuracy,  and 
user  acceptance.  To  resolve  the  problem,  Rome  Air  Development  Center  funded 
Threshold  Technology  in  1977  to  conduct  a series  of  studies  designed  to  shed 
empirical  light  on  the  subject. 

The  following  paragraphs  present  information  obtained  from  a preliminary, 
unapproved  copy  of  Threshold  Technology's  final  report. 

Threshold  Technology  conducted  two  data  entry  experiments  (Welch,  1977)  which 
varied  the  type  of  task,  data  entry  device,  feedback,  and  degree  of  hand 


occupation.  Using  48  volunteers  with  varying  experience  with  keyboard,  voice, 
and  Graf  pen  entry  devices.  Threshold  obtained  results  which  indicated  that: 

a.  For  simple  tasks  requiring  an  individual  to  copy  nuneric  data 
rapidly,  keyboard  entry  was  significantly  faster  and  more  accurate 
than  voice.  For  alphanumeric  data  entry,  keyboard  entry  was  faster 
but  less  accurate  than  voice  entry. 

b.  For  complex  tasks  requiring  cognitive  and  visual  effort,  voice 
entry  provided  higher  throughput  then  keyboard  entry,  particu- 
larly with  inexperienced  personnel,  because  of  its  ability  to 
free  the  eyes. 

c.  Voiced  feedback  (provided  by  a speech  synthesis  unit)  slowed  voice 
entry  because  most  subjects  waited  for  the  verbal  feedback  to  cease 
before  entering  additional  data. 

d.  The  requirement  to  correct  initial  recognition  errors  by  voice 
frequently  led  to  additional  errors  which  further  reduced  speed 
of  response. 

e.  Voice  data  entry  was  faster  than  other  entry  modes  when  the  hands 
were  occupied  for  a substantial  portion  of  the  total  entry  time. 

f.  Combining  verbal  feedback  (or  prompting)  with  visual  feedback 
(or  prompting)  did  not  facilitate  speed  or  accuracy  of  entry. 

Summing  up  these  results,  it  is  concluded  that  with  present  isolated- 
word  ASR  equipment  (which  requires  pauses  between  entries  and  which  has  a 2-3 
percent  error  rate  on  recognition)  ASR  is  not  clearly  the  faster  or  more 
accurate  data  entry  device.  Rather,  the  advantages  of  voice  entry  are  limited 
to  certain  types  of  tasks,  to  inexperienced  operators,  or  to  situations  where 
hands  or  eyes  are  not  free  for  other  forms  of  entry.  Under  the  pressure  of  a 
high  speed  entry  task,  the  correction  of  errors  made  during  initial  voice 
entry  often  led  to  more  errors  which  was  observed  to  "rattle"  the  subjects. 
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This  result  does  not  support  ASR  over  present  crew  station  entry  methods, 
particularly  in  combat  or  other  pressure-inducing  situations.  As  pointed 
out  by  Threshold,  a combination  of  entry  devices  is  likely  to  meet  all  the 
requirements  for  each  specific  task.  This  will  require  human  factors  studies 
and  trade-offs  to  optimize  overall  data  entry  design.  These  generalizations 
are  limited  to  the  condition  of  Threshold  Technology's  studies  and  should 
be  reexamined  with  tasks  more  relevant  to  aircrew  duties. 

2.1.4  TechnolocLv  Summary 

The  purpose  of  this  summary  is  to  assess  the  status  of  successful  applica- 
tion of  automatic  speech  recognition.  To  accomplish  this,  it  seems  useful 
to  review  all  of  the  advancements  described  in  the  review  of  applications 
(Section  3.0)  and  speculate  about  what  their  combination  might  be  capable 
of  producing.  For  this  conclusion,  the  following  dimensions  of  AST  reflect 
present  capabilities  as  verified  by  demonstration  or  test: 

a.  Type  of  Speech.  For  the  present  (and  probably  at  least  the  next 
five  years),  applications  will  be  confined  to  the  use  of  iso- 
lated word  devices  where  the  vocabulary  is  predefined,  rigorously 
adhered  to  by  users,  and  composed  of  discrete  words  or  phrases 
separated  by  short  (0,25  second)  pauses.  The  development  of 
limited  continuous  speech  recognition  (LCSR)  may  liberalize 
these  conditions,  allowing  perhaps  different  syntactical  arrange- 
ments of  key  words  and  eliminating  the  requirement  for  pauses 
between  words.  LCSR,  however,  will  still  require  a predefined 
vocabulary. 

b.  Vocabulary.  At  the  present,  isolated  word  devices  are  limited  to 
approximately  100  utterances  without  reducing  recognition  accu- 
racy. For  certain  applications,  the  vocabulary  can  be  constructed 
to  handle  a much  larger  number  of  items  (perhaps  500)  with  good 
recognition  accuracy  accomplished  by  ordering  and  by  sequencing 
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voice  entries  according  to  syntactical  rules  for  data  entry 
(e.g.,  systen  Identification  - switch  Identification  - value 
or  setting).  Wherry's  syntactical  handler  for  the  VRAS  not  only 
will  allow  expansion  of  the  vocabulary  but  will  allow  gi*eater 
flexibility  on  the  user's  part  by  allowing  different  syntactical 
strings  to  be  semantically  equivalent. 

c.  Recognition.  At  the  present.  Isolated  word  devices  are  capable 
of  95+  percent  recognition  accuracy  with  a vocabulary  of  100 
items.  However,  recognition  accuracy  alone  can  be  a misleading 
figure  because  the  false  alarm  rate  must  also  be  taken  into  ac- 
count. Extremely  high  recognition  accuracy  can  be  obtained  if 
one  is  willing  to  suffer  a high  false  alarm  rate  and/or  a high 
rejection  rate.  In  many  cases,  however,  a false  alarm  or  a re- 
jection is  as  harmful  to  system  goals  as  an  error  (which  is  to 
say  lower  recognition  accuracy). 

d.  User  Acceptance.  At  present,  most  applications  require  the  user 
to  train  (or  tune)  the  system  on  his  voice  characteristics.  This 
process  of  training  the  system  serves  two  purposes: 

1.  It  creates  reference  data  for  subsequent  recognition 
process,  and 

2.  It  trains  the  user  on  the  syntactical  requirements  of  the 
vocabulary. 

To  be  most  useful  for  the  first  purpose,  the  device  should  be 
trained  so  that  the  user  pronounces  each  vocabulary  item  as  if 
he  were  actually  using  it  (i.e.,  with  appropriate  voice  inflection, 
etc.)  in  the  system  context.  This  procedure  will  produce  greater 
fidelity  between  training  and  use,  and  thereby  facilitating  recognition 
accuracy. 
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Developments  such  as  Wherry's  syntactic  handler,  and  Dialog 
System's  speaker  independent  system  (although  untried  in  appli- 
cation) offer  promise  for  improving  user  acceptance  of  AST. 

Without  user  acceptance  based  on  actual  beneficial  performance, 

AST  is  just  another  new  technology. 

The  combination  of  new  developments  contributed  by  NADC,  NASA  Ames,  Logicon, 
Dialog  Systems,  and  other  agencies  provides  a powerful  new  technology  for 
man/machine  interface  designers.  With  careful  planning,  these  various  ad- 
vancements can  be  used  to  make  man's  communication  with  machine  natural  and 
efficient. 

2.2  Application  to  Selected  Crew  Station  Design 

It  was  anticipated  that  a large  number  of  AST  applications  and  requirements 
could  be  found  through  the  detailed  analysis  of  a typical  crew  station  design. 
Advanced  research  and  development  activities  in  the  areas  of  cockpit  and  crew 
station  design  had  already  demonstrated  the  technical  feasibility  of  utiliz- 
ing AST  to  assist  aircrews  in  their  operational  duties  (e.g.,  VRAS).  Speech 
technology  could  provide  additional  modes  for  information  transmission  and 
could  thereby  unburden  the  operator  during  periods  of  peak  workload  by  ex- 
ploiting his  basic  propensity  for  hearing  and  speaking  while  engaged  in  vis- 
ual or  psychomotor  activities. 

The  fundamental  approach  to  this  task  was  to  first  choose  a typical  crew 
station  system  to  use  as  a baseline  from  which  to  gather  actual  crew  task 
data.  The  next  steps  were  to  obtain,  analyze,  and  rate  the  task  data  for 
potential  applicability  to  AST  payoff.  Matrices  were  then  developed  of 
these  tasks  versus  the  particular  crew  stations  and  subsystems  with  which 
the  tasks  were  associated. 

2.2.1  Baseline  Mission  Selection 

Because  the  task  of  analyzing  all  available  aircraft  crewstations  for  possible 
AST  applications  was  obviously  impossible,  it  was  proposed  that  only  a limited 
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number  be  examined  In  sufficient  detail  to  determine  applications.  The  selec- 
tion of  a single  crew  station  model  or  baseline  system  was  made  at  the  pro- 
gram Kickoff  Meeting.  The  P-3C  Antisubmarine  Warfare  (ASW)  aircraft  was  chosen 
on  the  basis  of: 

a.  The  wide  variety  of  operator  tasks  associated  with  all  of  the 
operator  workstations  aboard  the  P-3C. 

b.  The  high  probability  that  the  P-3C  will  still  be  in  the  opera- 
tional inventory  when  many  AST  applications  are  developed  to  the 
point  of  requiring  operational  evaluation.  This  factor  is  of 
benefit  in  two  ways: 

1.  Baseline  data  without  AST  will  still  be  easily  available,  and 

2.  The  same  P-3C  subsystems/components  may  be  used  for  testing 
and  comparison  throughout  the  evolution  of  the  AST  applications 
project. 

c.  There  is  at  present  a large  amount  of  P-3C  operator  task  data  which 
is  relatively  available.  These  data  consist  of: 

1.  The  NATOPS  manuals  for  each  of  the  crew  positions  (NAVAIR 
01-75PAC-1.X  series), 

2.  Courseware,  Incorporated  volumes  (entitled  "Job  Analysis 
Document")  for  each  of  the  more  significant  crew  positions, 

3.  Direct  interviews  with  P-3C  operators,  and 

4.  The  P-3C  simulation  system  (as  well  as  other  P-3C  support 
operations)  at  NADC  and  other  facilities. 

d.  All  other  systems  examined  had  either  too  few  operator  stations  or 
were  scheduled  to  be  available  too  far  in  the  future  to  have  any 
available  data  for  comparison  at  this  time  (e.g.,  VPX). 

The  use  of  the  P-3C  baseline  was  limited  to  an  examination  of  only  the  on- 
station  portion  of  the  mission.  This  limit  had  to  be  made  because  of  pro- 
gram time  and  manhour  availability.  It  should  be  noted  that  AST  applications, 
which  may  be  unique  to  other  mission  segments,  were  not  examined  in  this 
study  program. 
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For  example,  a major  crew  task  procedure  during  the  preflight  portion  of  the 
mission  is  the  performance  of  various  checklists.  This  procedure  would  lend 
itself  extremely  well  to  the  application  of  AST;  however,  it  was  not  con- 
sidered as  a part  of  this  program  due  to  the  mission  on-station  limit  for 
crew  task  analysis.  A further  limit  to  the  P-3C  baseline  was  in  the  number 
of  crew  stations  analyzod.  Only  the  most  significant  crew  stations  were  in- 
cluded. These  crew  stations  were: 

a.  Pilot/Copilot, 

b.  Tactical  Coordinator  (TACCO), 

c.  Navigation/Communications  Officer  (NAV/COMM), 

d.  Sensor  Station  Operators  1 and  2 (SS-1,  SS-2),  and 

e.  Sensor  Station  Operator  3 (SS-3). 

Although  other  crew  stations,  namely  the  Ordnance  Station,  Technician  Station 
and  Flight  Engineer  could  have  been  analyzed,  the  time  and  budget  available 
would  not  allow  this  effort  in  comparison  to  the  other  program  tasks. 

2.2.2  P-3C  Baseline  Description 

The  following  description  of  the  P-3C  aircraft  and  mission  may  be  of  assistance 
in  understanding  the  AST  applications  selection  process. 

The  P-3C  is  a four-engine,  low-wing  aircraft  designed  for  patrol  and  anti- 
submarine warfare.  Distinguishing  features  of  the  airplane  include  advanced 
submarine  detection  gear  including  computer  interfacing  of  the  detection  gear, 
the  ordnance  system.and  the  armament  systems.  The  model  is  readily  identified 
by  the  camera  installed  in  the  lower  section  of  the  forward  radome,  the  in- 
stallation of  sonobuoy  chutes,  visible  in  the  lower  aft  fuselage  of  the  air- 
plane and  three  additional  small  windows  on  the  right  side  of  the  fuselage. 
Provisions  are  installed  for  carrying  a streamlined  low-light-level  television 
camera  pylon  and  pod  on  a wing  station.  Also  provisions  for  carrying  a stream- 
lined ECM  pod-pylon  assembly  are  installed  on  a wing  station. 

The  primary  mission  of  the  P-3C  aircraft  is  detection,  localization,  surveil- 
lance, and  attack  of  targets  that  pose  potential  military  threat.  Satisfac- 
tory pursuit  of  this  mission  is  realized  through  the  two  phases  of  contact 
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development  and  contact  refinement.  Each  creMmember  pleys  a vital  role 
in  support  of  this  mission,  and  the  P-3C  aircraft  is  designed  and  built 
to  be  operated  as  an  integrated  team  effort. 

The  tactical  coordinator  (TACCO)  is  responsible  for  the  tactical 
portion  of  the  flight  mission  and  will  coordinate  the  functions  of  the 
entire  flight  crew.  The  pilot,  as  aircraft  commander.  Is  responsible 
for  the  flight  crew  being  in  their  assigned  positions  for  takeoff  and  land- 
ing* including  ditching  in  an  emergency.  Each  crewmember  has  indi- 
vidual responsibilities  and  duties  as  described  in  the  following  sections. 
Additional  duties  and  responsibilities  are  assigned  by  the  pilot  and 
TACCO  as  necessary. 

Each  crewmember  shall  possess  a thorough  knowledge  of  the  equipment  at  his 
station,  plus  a familiarity  with  equipment  used  by  other  crewmen,  so  that 
he  can  assume  other  duties  in  an  emergency  and  facilitate  normal  crew  coordi- 
nation. Each  crewmember  is  expected  to  be  thoroughly  familiar  with  safety 
and  survival  equipment  in  the  aircraft  and  to  be  completely  knowledgeable 
in  the  use  and  wearing  of  his  personal  equipment. 

2.2.3  Crew  Task  Analysis 

The  next  step  in  the  crew  station  design  task  was  to  obtain  P-3C  crew  task 
data.  Much  of  these  data  were  available  in  the  NATOPS  manuals  (series  NAVAIR 
01-75PAC-1 .X).  The  manuals  were  examined  and  found  to  be  useful  in  providing 
data  as  to  the  very  general  task  duties  of  each  crewman.  They  were  also  ex- 
cellent at  providing  the  details  of  several  frequently  used  crew  procedures. 
The  following  position  descriptions  were  taken  from  the  NATOPS  manuals: 

a.  Pilot/ Copilot.  When  the  pilot  is  assigned  the  duties  of  mission 
commander,  he  shall  be  responsible  for  all  phases  of  the  assigned 
mission.  He  shall  direct  a coordinated  plan  of  action  and  shall 
be  responsible  for  the  effectiveness  of  the  flight.  He  shall  be 


responsible  for  the  crew  preparation  for  takeoff,  and  for  take- 
off at  the  scheduled  time.  He  shall  gather  and  evaluate  reports 


on  the  aircraft  and  equipment  and  direct  preparation  for  flight  as 
necessary.  Further,  the  pilot  acting  as  mission  commander  shall 
sanction  armament  selection  and  release. 

As  the  patrol  plane  coimiander,  the  pilot  is  responsible  for  the 
effectiveness  of  the  aircraft  and  crew  for  all  matters  affecting 
safety  of  flight.  Prior  to  starting  engines,  before  taxi,  before 
takeoff,  and  at  other  scheduled  times  inflight,  he  shall  call  for 
the  appropriate  checklist  to  be  read  by  the  copilot,  and  shall  re- 
spond as  necessary.  As  aircraft  commander,  he  shall  coordinate 
ASW  tactics  with  the  TACCO  and  fly  the  aircraft  as  directed  by  the 
flight  director  indicator/horizontal  situation  indicator  and  tacti- 
cal situation  display  in  the  prosecution  of  the  mission  problem. 

The  pilot  will  stabilize  the  tactical  plot  via  the  on-top  function. 

He  will  evaluate  the  tactical  plot  and  coordinate  with  the  TACCO 
the  updating  of  the  plot.  The  pilot  will  also  enter  visual  contact 
data  into  the  computer  in  support  of  the  mission. 

The  copilot  shall  assist  the  pilot  in  preparing  the  crew  for  flight 
and  in  ascertaining  readiness  for  flight  of  the  aircraft  and  air- 
craft systems.  He  shall  read  the  checklist,  as  required  by  the 
flight  mission.  He  will  pilot  the  aircraft  at  all  times  the  pilot 
is  away  from  his  station.  The  copilot  function  is  specifically 
patterned  as  a safety  back-up  for  the  pilot  throughout  the  entire 
flight.  In  this  capacity  he  shall  offer  constructive  comments  and 
recommendations  as  necessary  throughout  the  mission  in  order  to 
maintain  the  safest  possible  and  most  effective  flight  environment. 
The  copilot  shall  call  out  all  altitudes,  airspeeds,  and  angles  of 
bank  as  directed  by  the  pilot,  or  the  minimum  safe  altitude/air- 
speed for  the  mission.  He  may  also  be  required  to  release  stores, 
read  checklists,  operate  the  cameras,  provide  ship  rigging  informa- 
tion to  the  TACCO  or  NAV/COMM  for  computer  entry  and  any  other  duties 
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as  directed  by  the  pilot.  During  the  times  the  copilot  is  in 
control  of  the  aircraft,  his  coordination  of  crew  duties  shall 
be  the  same  as  for  the  pilot. 

b.  TACCO.  The  TACCO's  function  is  to  employ  appropriate  tactics 
and  procedures  to  most  effectively  carry  out  the  mission  of  the 
aircraft  and  its  crew.  He  will  initiate  a coordinated  plan  of 
action  for  all  tactical  crewmembers  and  continuously  monitor,  re- 
view and  revise  the  plan  as  the  situation  dictates.  He  will  make 
decisions  regarding  search  and  kill,  stores  selection  and  release. 

He  shall  ensure  the  accurate  completion,  collection  and  disposi- 
tion of  required  magnetic  tapes,  logs  and  records. 

The  deployment  of  search  stores  is  determined  by  the  TACCO,  and 
is  normally  accomplished  by  the  computer.  The  ordnancemarv  when 
directed  by  either  the  TACCO  or  the  PILOT,  may  select  and  launch 
a store  either  manually  from  a pre-loaded  SLT  (sonobuoy  launch 
tube)  or  PSLT  (pressurized  sonobuoy  launch  tube)  or  in  the  event 
of  complete  equipment  malfunctions,  through  the  free  fall  chute. 

Kill  stores  are  selected  in  conjunction  with  the  pilot  by  the 
TACCO. 

The  TACCO  shall  coordinate  the  efforts  of  all  tactical  crewmembers 
advising  of  the  possibility  of  contact  as  well  as  informing  them 
of  surface  traffic,  and  the  spatial  sonobuoy  distributions.  The  TACCO  will 
ensure  that  the  proper  EMCON  (emission  control)  condition  is  maintained. 

i 

1 

c.  NAV/COHM.  It  is  the  responsibility  of  the  navi gation/comnuni cations  I 

officer  (NAV/COMM)  to  maintain  an  accurate  record  of  present  and 

past  positions,  to  insert  navigation  fly-to-points,  update  geogra- 
phical position,  transmit  tactical  messages  as  authorized  for  re- 
lease by  the  aircraft  commander,  set  up  radio  equipment  before  flight. 
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and  maintain  a record  of  the  flight.  The  NAV/COMM  is  responsi- 
ble for  navigating  the  aircraft  to  the  specified  operational 
area  and  transmitting  aircraft  position  reports  in  accordance 
with  directives  promulgated  by  the  operational  comnander.  The 
NAV/COMM  shall  provide  data  link  assistance  as  directed  by  the 
TACCO.  The  NAV/COMM  shall  also  monitor  navigation  systems  in 
use.  The  TACCO  shall  be  advised  of  navigation  system  failures. 

d.  Sensor  1 and  2.  It  is  the  responsibility  of  the  acoustic  sensor 
operators  (SS-1  and  SS-2)  to  detect  and  classify  contact  data. 

The  audio  information  is  recorded  for  subsequent  mission  recon- 
struction. The  determination  of  sono  target  evaluation  will  be 
in  close  concert  with  TACCO  for  the  determination  of  buoy  types, 
RF's,  target  signatures,  surface  traffic,  and  the  aggregate  ASW 
environment. 

e.  Sensor  3.  The  Sensor  Station  3 (SS-3)  operator's  function  is  to 
determine  the  position  of  a submarine  by  detecting  changes  in 
earth's  Magnetic  Field  caused  by  the  submarine's  hull,  to  detect 
and  analyze  targets  of  military  significance  and  provide  radar 
intercept  and  navigation  information  to  the  Plane  Commander.  He  al- 
so will  challenge  the  identity  of  these  targets.  In  addition,  the 
Sensor  Station  3 operator  will  passively  detect  targets  of  mili- 
tary significance  using  Low  Light  Level  Television  (LLLTV)  and 

ECM  (electronic  counter  measures). 

During  magnetic  and  submarine  anomaly  detection  (MAD  and  SAD), 
the  TACCO/PILOT  notifies  the  SS-3  operator  that  the  airplane  is 

I approaching  the  possible  target  location.  The  SS-3  operator  will 

I 

j announce  the  presence  of  targets  in  the  area  as  directed  by  the 

I ECM/LLLTV  equipment.  The  SS-3  operator  will  announce  bearing  and 

range  of  targets  orally  and/or  by  the  keyboard  functions,  detected 

i 

( by  search  radar  set  and  challenged  by  radar  recognition  set.  The 

TACCO  will  specify  the  operational  employment  of  the  radar  and 
challenging  by  radar  recognition  set.  (EMCON) 
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In  order  to  properly  analyze  operator  tasks,  data  were  needed  as  to 
the  sequence,  frequency,  and  criticality  of  the  on-station  crew  pro- 
cedures. Very  little  of  this  type  of  information  was  provided  in  the 
NATOPS  manuals.  Fortunately,  this  information  had  just  been  obtained 
and  documented  in  several  volumes  by  Courseware,  Incorporated  for  the 
Instructional  System  Development  (ISD)  Team.  This  team  was  formed  to 
develop  training  courses  for  the  Readiness  Training  Squadrons  VP-30  at 
N.A.S.,  Jacksonville,  Florida  and  VP-31  at  N.A.S.,  Moffett  Field,  Cali- 
fornia. Courseware  obtained  these  data  from  Navy  training  experts  who 
used  existing  documentation  and  review  by  their  colleagues.  Courseware 
then  conducted  a survey  to  gather  data  from  approximately  two  dozen  crew- 
men for  each  of  the  P-3C  crew  positions.  These  data  consisted  of  the 
crewmen's  opinions  as  to  how  relevant,  critical,  frequent,  and  diffi- 
cult each  of  the  mission  tasks  was.  The  information  as  to  criticality 
and  frequency  was  particularly  useful  to  this  study  program.  These  rat- 
ings, which  were  provided  for  the  purpose  of  ISD,  were  of  invaluable  use 
to  the  prioritization  of  crew  station  tasks  in  relation  to  AST  applications. 

The  next  subtask  to  be  performed  for  the  crew  station  analysis  was 
to  obtain  sufficiently  detailed  task  data  to  correlate  to  potential  AST  appli- 
cation. The  Courseware  data  was  generally  one  level  of  detail  too  high  or 
too  gross  to  be  useful.  The  level  of  detail  needed  had  to  be  obtained  from 
the  NATOPS  manuals  and  Boeing  and  Navy  personnel  who  had  direct  experience 
with  the  P-3C  crew  tasks.  Interviews  were  conducted  with  these  persons  for 
the  purpose  of  filling  in  and  adding  to  the  Courseware  document  task  lists. 

In  addition  to  the  Boeing  personnel  interviews,  numerous  Moffett  Field  per- 
sonnel were  interviewed  as  to  their  opinions  and  experience  in  relation  to 
crew  tasks.  To  a limited  extent,  the  Moffett  Field  simulator/training  hard- 
ware was  also  examined  as  was  the  actual  P-3C  aircraft. 

Detailed  task  list  worksheets  such  as  that  shown  in  Figure  2.2-1  were  con- 
structed for  each  of  the  crewstations  analyzed.  The  most  tasks  analyzed, 
approximately  320,  were  for  the  pilot/copilot  workstation. 
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POSITION:  P-3C  PILOT  DETAILED  TASK  RATINGS 


Figure  2.2- 1:  Typical  Detailed  Task  Worksheet 


p 
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In  addition  to  the  subtask,  frequency,  and  criticality  data,  information 
as  to  the  subsystem  with  which  each  subtask  was  performed  and  the  general 
generic  task  category  were  included  on  the  task  list  worksheets.  The  use 
of  the  subsystem  and  generic  task  data  is  explained  in  Section  2.2.5  of 
this  report. 


2.2.4  Rating  Process 


The  next  subtask  in  the  total  crew  station  design  analysis  procedure  was 
to  rate  each  of  the  crew  tasks  in  order  to  evaluate  and  select  those  most 
amenable  to  AST  application.  ^After  consideration  of  the  variables  that 
affect  the  application  of  voice  technology  to  crew  station  design,  a four 
factor  rating  system  was  dev al oped.  The  four  factors  are  listed  and  de- 
scribed below.  - '■ 


2. 2.4.1  Technical  Feasibility  Factor. 


Could  be  imolemented: 

a.  Immediately:"  i .e. , requires  an  isolated  word  device,  limited  vocabulary, 
rigid  syntax,  speaker  training  (rating  =1). 

In  two  (or  more)  years;  i.e.,  requires  limited  continuous  speech 
recognition,  syntactic  handler,  speaker  adaptation  with  little 
training,  limited  vocabulary  (rating  = 2). 


b. 


c. 


In  four  (or  more)  years;  i.e.,  requires  continuous  free  speech 
recognition,  unconstrained  vocabulary,  independent  speaker  (rating  = 
3). 


d.  Probably  never;  cannot  be  performed  effectively  by  speech,  requires 
100  percent  accuracy  in  real  time  (rating  = 4). 


2. 2. 4. 2 Utility  Factor. 


a.  Voice  benefits  the  crew  member;  i.e.,  crew  task  is  amid  heavy 
workload,  hand,  eyes,  or  attention  used  for  concurrent  tasks, 
visual  channel  is  overloaded  (rating  =1). 
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b.  Voice  application  is  equivalent  to  present  system;  i.e.,  no  obvious 
advantage  for  applying  speech  considering  workload,  time  demands, 
etc.  (rating  = 2). 

c.  Voice  application  creates  a disadvantage;  i.e.,  task  environment 
is  noise,  task  is  highly  critical,  voice  already  is  used  for  other 
tasks  demanding  100  percent  accuracy  with  no  time  for  feedback 
(rating  = 3). 


a.  Low  requirements;  i.e.,  80  percent  recognition  accuracy  is  suffi- 
cient on  first  attempt  with  enough  time  for  verification/correction 
process  (rating  = 1). 

b.  Requirements;  i.e.,  90  percent  accuracy  with  moderate  time  pres- 
sure (rating  = 2). 

c.  High  requirements;  i.e.,  98  percent  accuracy  required  with  vir- 
tually no  time  for  error  recovery  (rating  = 3). 

2. 2. 4. 4 Unassessed  Variables  Factor.  The  unassessed  variables  include  air- 
craft noise,  job  pressure,  cross  conversation,  lengthy  mission  segments,  etc. 

a.  None  (rating  = 1) 

b.  Few  (rating  = 2) 

c.  Some  (rating  = 3) 

d.  Many  (rating  = 4) 

2. 2.4. 5 Rating  Assignment  Procedure.  Using  the  preceding  rating  structure, 
each  task  was  reviewed  utilizing  the  four  dimensional  factors  (i.e.,  tech- 
nical feasibility,  utility,  time/accuracy  requirements,  and  likelihood  of 
unknowns).  Thus,  each  task  was  assigned  a four  digit  code  of  numbers  corres- 
ponding to  task  factors  or  requirements.  For  example,  a TACCO  tray  switching 
task  might  be  given  a 1 2 2 1 code,  indicating  that  it  can  be  implemented  now 
by  voice  with  no  real  gain  over  the  button  push,  with  little  pressure,  and 
with  no  unforeseen  variables. 
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Overall  tasks  were  assigned  four  digit  codes,  and  the  four  digit  code  was 
reduced  to  a one  digit  code  (1-5)  corresponding  to  AST  pay-off  at  this 
time.  The  five  codes  were  as  follows: 

a.  High  pay-off  (codes  of  1111  or  1112)  = (Code  1). 

b.  Some  pay-off  (Codes  of  2122,  2112,  or  2111)  = (Code  2). 

c.  Questionable  pay-off  (Codes  2222,  1211,  2232,  1221,  1212  or 

2233)  = (Code  3). 

d.  Very  low  pay-off  (Codes  3233  or  2222)  = (Code  4), 

e.  No  pay-off  (Codes  4333,  4334)  = (Code  5). 

When  each  task  was  given  a single-digit  AST  rating,  the  criticality  and 
frequency  ratings  from  the  Courseware  documents  were  applied  to  the  AST 
rating  to  arrive  at  an  overall  AST  rating.  The  following  guidelines  were 
applied  to  determine  the  overall  AST  rating: 

a.  If  criticality  is  high,  frequency  is  high  or  moderate,  and  AST 
is  moderate  (Code  2 or  3),  then  decrease  the  overall  AST  rating 
by  one  digit. 

b.  If  criticality  is  moderate,  frequency  is  high  or  moderate,  and 
AST  is  moderate  (Code  2 or  3),  increase  the  AST  rating  by  one 
digit. 

c.  If  other  situations  exist,  then  set  the  overall  AST  rating  at 
a level  equivalent  to  the  initial  AST  rating. 

For  the  purpose  of  providing  summary  presentation  material  and  drawing  gen- 
eral conclusions  as  to  types  of  AST  application  payoff  areas,  all  tasks  were 
converted  to  a matrix  format  described  in  Section  2.2.5  (i.e.,  placed  into 
generic  task  by  subsystem  cells).  The  AST  codes  for  all  tasks  within  a ma- 
trix cell  then  were  treated  statistically  to  determine  a single  AST  code 
(median)  for  each  matrix  cell  which  reflected  the  central  tendency  of  that 
cell.  Using  standard  rounding  procedures,  the  median  cell  AST  rating  was 
converted  to  a code  as  follows: 


a.  AST  rating  1.2-1. 4 = 1 

b.  AST  rating  1.5-2. 4 = 2 

c.  AST  rating  2. 5-3.4  = 3 

d.  All  else  = Blank,  or  "C"  to  indicate  relative  continuous 

use  by  crewmen 

In  terms  of  a plan  for  voice  application,  a "1"  would  represent  inriediate 
application  with  good  pay-off;  a "2"  would  indicate  near-term  application 
with  some  pay-off,  and  a "3"  would  indicate  long-term  application  with 
questionable  pay-off.  The  "C",  for  continuous  use,  indicates  that  the  partic- 
ular subsystem-generic  task  cell  combination  is  used  relatively  continuously, 
but  on  a routine  and  low  priority  basis.  These  crew  tasks  tend  to  be  monitor- 
ing functions  performed  on  a routine  station  keeping  basis. 

The  use  of  the  lower  ratings  (i.e.,  2 and  3)  does  not  necessarily  mean  that 
the  particular  task  does  not  have  high  potential  payoff.  The  lower  rating 
may  mean  that  the  task  is  a candidate  for  6.1  or  6.2  program  funding  in  order 
to  solve  a problem  of  continuous  speech  recognition,  speaker  adaptation,  or 
large  vocabularies.  In  order  to  give  visibility  to  certain  tasks  which  re- 
ceived low  ratings  only  from  the  technical  feasibility  factor,  they  are  pre- 
sented in  a special  separate  pilot  matrix  (Table  2.2-9)  in  the  following 
2.2.5  section  (Matrices  Development).  This  section  contains  matrices  for 
each  crew  position  to  show  general  areas  of  AST  potential  payoff  in  relation 
to  generic  crew  tasks  and  subsystems. 

2.2.4. 6 Rating  Assumptions.  This  entire  rating  process  was  based  on  the 
following  assumptions: 

a.  Crew  members  can  be  trained  to  use  speech  input  for  up  to  six 
consecutive  hours  while  maintaing  reasonable  accuracy. 

b.  The  AST  system  operates  in  real-time. 

c.  Cost  is  not  considered  as  a factor,  at  this  time;  that  is,  revision 
of  displays  and  computer  programs  is  allowable. 

d.  Users  are  cooperative  and  resources  are  committed  to  ensuring 
user  acceptance  through  task  and  system  design. 
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2.2.5  Matrices  Develot 


As  previously  indicated  in  Section  1.3,  preliminary  crew  station  design 
data  were  presented  at  the  July  5 and  6 Apportionment  Review.  A summary  of 
the  crew  station  task  analysis  was  presented  in  the  form  of  a matrix  for 
the  two  crew  stations  examined  (Pilot  and  TACCO).  Because  of  their  sumnary 
form,  no  information  as  to  specific  AST  application  projects  could  be  ob- 
tained from  these  matrices.  However,  the  matrices  did  point  to  general 
areas  as  to  types  of  generic  crew  tasks  and  P-3C  subsystems  which  would 
tend  to  be  amenable  to  AST  applications.  Following  the  Apportionment  Re- 
view, crew  station  design  summary  matrices  were  developed  for  the  other 
crew  stations  and  reworked  (with  the  addition  of  newer  task  analysis  data) 
for  the  first  two  crewmen. 


The  first  steps  in  the  development  of  the  AST  pay-off  area  matrices  were 
to  decide  both  the  level  of  crew  tasks  and  method  of  aircraft  subsystem 
allocation.  The  longer  the  lists  of  crew  task  categories  and  subsystem 
categories,  the  more  detailed  the  information  that  could  be  obtained  from 
the  cell  intersection  between  these  two  parameters.  However,  it  was  not 
the  intent  of  these  matrices  to  provide  detailed  information  as  to  specific 
AST  pay-off  areas.  Specific  applications  could  come  only  from  detailed  analy- 
sis of  crew  procedures  and  equipment  functions.  Since  the  matrices  were  in- 
tended only  for  overview  information,  it  was  decided  to  provide  sufficient 
task  and  subsystem  comparison  data  for  a one  page  matrix  form.  This  meant  that  the 
crew  tasks  should  be  divided  into  eleven  generic  categories  and  the  subsystems 
into  thirteen  categories  with  the  ASW  subsystem  divided  into  ten  subcategories. 

This  subcategory  division  was  due  to  the  desirability  for  relatively  more  visi- 
bility to  this  subsystem  because  of  its  extensive  use  by  all  crew  members.  The 
following  Table,  2.2-1,  lists  each  of  the  selected  generic  task  categories,  along 
with  a brief  definition  of  the  particular  generic  task.  Table  2.2-2  is  a 
list  of  P-3C  subsystems  and  subsystem  descriptions  taken  from  the  NATOPS 
manuals.  In  anticipation  of  the  matrices  construction,  each  crew  task  was 
analyzed  and  categorized  as  to  the  type(s)  of  generic  task(s)  and  subsystem(s) 
involved  in  the  task  accomplishment.  These  data  were  indicated  by  their 
abbreviations  (which  are  in  parenthesis)  on  the  task  analysis  worksheet  form 
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Table  2.2-1  Generic  Task  Definitions 


On  Top  Pos  Ind  (OTPI)  - Group  enables  aircraft  to  home  on  a selected  SONO  (VHP  receiver  and  UHF 

direction  finder) 

BY  ft  Sea  Noise  (BT  &SN)  - Bathythermograph  sonobuoys,  chart  recorder 

MAD  - Magnetic  Anomaly  Detection:  magnetic  detecting  set,  submarine  anomaly  de- 

tecting group,  magnetic  compensator  group,  etc. 


F “ 

(reference  Figure  2.2-1).  After  the  tasks  were  rated,  they  were  sunriarized 
along  with  the  generic  task  and  subsystem  relationships  for  incorporation 
into  the  matrices  presented  in  Tables  2.2-3  through  2.2-7.  Table  2.2-8  is 
a composite  for  all  of  the  crew  positions  of  all  the  data.  The  composite 
table  does  not  include  the  continuous  use  code. 

Because  there  was  some  concern  that  the  use  of  the  feasibility  factor  to 
rate  potential  payoff  areas  might  mask  the  choice  of  research  projects,  a 
second  pilot  matrix  was  constructed  without  the  feasibility  factor.  Table 
2.2-9  is  this  matrix.  Comparison  of  this  table  with  Table  2.3-3,  which  in- 
cluded the  use  of  the  feasibility  factor,  shows  several  differences.  The 
additions  to  Table  2.2-9  are  as  follows: 

a.  One  high  payoff  area  is  added  to  the  Monitor  Indicator  row 
under  Flight  Instruments.  This  row  suggested  a project  for 
a pilot  cue/alert  system. 

b.  Two  high  payoff  areas  were  added  to  the  Determine  Solution 
row  under  Fuel  and  Crew. 

c.  The  low  payoff  rating  for  the  Receive  Data  row  under  Coimuni- 
cations  changed  to  a high  payoff  rating.  These  data  along 
with  the  previous  data  suggested  a voice  data  retrieval  system. 

d.  Most  of  the  low  and  medium  payoff  areas  for  the  Activate  Con- 
trols row  changed  to  high  payoff  areas.  These  areas  suggest 
both  the  need  for  a checklist  verification  system  and  a pilot 
VIS,  in  general. 

e.  The  Adjust  Controls  row  payoff  area  under  Communications  is 
changed  from  medium  to  high,  and  the  low  payoff  area  for  the 
Perform  Maneuver  row  under  Flight  Instruments  is  added. 

Although  the  construction  of  these  matrices  was  helpful  in  selecting  candi- 
date applications/projects  to  be  presented  in  the  Program  Plan,  the  total 
process  of  generating  the  data  to  go  into  the  matrices  proved  to  be  even 
more  helpful.  The  following  section  describes  alternative  methodologies 
for  determining  and  identifying  crew  station  design  AST  applications/projects. 

i 
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Table  2.2-3:  Crew  Station  Design  Voice  Technology  Potential  Payoff  Aram 
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Table  2.2-4:  Crew  Station  Design  Voice  Technology  Potential  Payoff  Areas 
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Table  2.2-7:  Crew  Station  Design  Voice  Technology  Potential  Payoff  Aram 
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2.2.6  Alternative  Applications/Projects  Identification  Methodologies 


1 


! 

i Rather  than  determine  applications/projects  from  crew  task  analysis,  the 

hardware  used  in  the  accomplishment  of  crew  procedures  could  be  examined 
i directly.  Certain  displays  and  controls  tend  to  imply  the  possible  addi- 

tional use  or  replacement  use  of  AST  devices  for  specific  tasks.  For 
example,  cockpit  annunciator  lights  imply  the  use  of  voice  generated  ad- 
visory data.  Multifunction  discrete  switches  may  be  effectively  replaced 
by  automatic  speech  recognition  equipment.  The  use  of  paper  and  pencil 
for  procedural  logging  of  discrete  written  words  or  phrases  implies  the 
use  of  automatic  speech  recognition.  Another  approach  would  be  to  examine 
crew  tasks  to  determine  all  of  those  which  require  a high  frequency  of  com- 
bined hand  and  eye  use.  AST  could  be  used  to  relieve  much  of  the  hand  task 
loading  and  its  use  should  therefore  be  investigated  as  an  applications/ 
project.  Several  potential  applications/projects  which  may  be  categorized 
crew  station  design  related  were  suggested  by  the  task  analysis  process 
directly  before  building  the  generic  task/subsystem  matrix.  Several  crew 
station  design  applications/projects  were  also  suggested  from  the  technology 
review  process  (Reference  Section  2.1). 

2.3  Significant  Additional  Application  Areas 

Prior  to  the  initiation  of  work  on  this  study  program,  it  was  anticipated 
that  all  the  proposed  applications/projects  could  be  derived  from  (and  cate- 
gorized accordingly)  the  three  areas  of  crew  station  design,  performance 
measurement,  and  training.  However,  two  additional  categories  are  approp- 
riate to  add  because  certain  of  the  proposed  applications/projects  do  not 
lend  themselves  to  any  particular  one  of  the  three  previously  indicated 
categories.  The  two  additional  categories  are  research  and  development  and 
maintenance.  The  research  and  development  category  pertains  to  those  pro- 
jects which  would  be  related  equally  to  each  of  the  other  categories  and 
would  generally  be  required  to  proceed  the  conduct  of  other  projects.  The 
maintenance  category  was  provided  for  one  project  which  pertains  to  both 
crew  station  design  and  training.  The  following  sections  present  general 
information  on  the  performance  measurement  and  training  aspects  of  AST. 
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2.3.1  Perfonnance  Measurement 


The  application  of  speech  technology  has  significant  implications  in  the 
area  of  operator  performance  measurement.  A basic  tenet  of  quantitative 
measurement  schemes  is  that  any  activity  which  can  be  objectively  observed 
can  also  be  measured.  The  use  of  speech  technology  can  serve  to  increase 
the  number  and  types  of  measurable  responses  which  until  now  have  not  been 
easily  observable  events.  The  utility  of  speech  technology  may  have  a sig- 
nificant impact  on  performance  measurement  capabilities  by  expanding  the 
measurement  domain  to  include  variables  and  parameter  sets  not  previously 
available  for  observation  in  any  precise  manner. 

In  order  to  determine  specific  possible  AST  performance  measurement  pro- 
jects, the  crew  station  analysis  task  was  reexamined.  In  addition  to  being 
examined  for  basic  performance  measurement  applications,  these  tasks  were 
examined  in  terms  of  characteristics  such  as  possible  performance  measure- 
ment parameters  and  conditions  and  the  applications  project  evaluation 
methodology. 

The  evaluation  for  the  above  performance  measurement  characteristics  has 
wider  application  than  just  to  proposed  performance  measurement  projects. 

The  end  product  of  this  study  program  was  a plan  which  contains  detailed 
methodology  as  to  what  AST  projects  should  be  developed  in  the  areas  of 
performance  measurement,  crew  station  design,  and  training.  Crew  station 
design  and  training  applications  needed  to  be  examined  in  terms  of  what 
performance  measurement  data  could  be  provided  by  their  test  and  evaluation. 
Figure  2.3-1  illustrates  the  anticipated  performance  measurement  data  avail- 
able for  each  of  the  projects/applications. 

2.3.2  Trai ni nq 

i The  application  of  AST  to  different  areas  of  training,  particularly  synthe- 

tic training  with  simulators  and  cockpit  procedures  trainers,  can  facilitate 
the  cost  effectiveness  of  present  training.  The  increased  cost  effective- 
ness can  be  accomplished  by: 
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MEASUREMENT  CONDITIONS  AND  PARAMETERS 
WITHOUT  AST 
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AST  USED  IN  PARALLEL  TO  EXISTING  SYSTEM  FOR  SELECTED  PREVIOUS  TASKS 


! 


a.  reducing  training  manning, 

b.  facilitating  the  roles  of  training  personnel  by  automating 
instructor  functions, 

c.  providing  training  where  it  is/has  been  difficult  to  schedule 
training  before  because  of  crew  availability  or  skill  level, 
and 

d.  providing  for  objective  measurement  of  training  and  transfer 
of  training. 

To  derive  specific  applications  of  AST  in  the  area  of  training,  team  train- 
ing environments  (such  as  that  for  the  P-3C)  were  considered.  This  coincides 
with  th  Fleet  Readiness  Squadron  (FRS,  formerly  called  RAG)  level  of  train- 
ing. Projects  were  identified  that  address  long  standing  problems  in  team 
training. 

2.4  Applications  Assessment 

The  accomplishment  of  the  preceding  analysis  study  tasks  (crew  station  de- 
sign, performance  measurement,  and  training)  resulted  in  five  separate  cate- 
gories or  lists  of  possible  AST  applications/projects  which  might  be  pursued 
in  accordance  with  the  recommendations  provided  in  the  Program  Plan.  Table 
2.4-1  lists  each  of  these  possible  projects  divided  according  to  the  AST 
category  from  which  they  were  derived.  As  can  be  seen  from  the  table,  the 
majority  of  the  proposed  projects  were  in  the  area  of  crew  station  design. 
Rather  than  propose  all  of  these  projects/activities  for  eventual  investi- 
gation or  development,  it  was  deciced  at  the  program  kickoff  meeting  to  in- 
clude a project  assessment  portion  of  the  total  task  effort.  Each  of  the 
activities/projects  was  compared  on  the  basis  of  certain  characteristics. 

They  were  numerically  rated  and  rank  ordered  to  determine  the  most  promis- 
ing activities/projects  for  incorporation  into  the  Program  Plan.  The  follow- 
ing sections  describe  the  details  of  this  assessment  process. 

2.4.1  Assessment  Factors 

The  characteristics  chosen  to  evaluate  the  activities/projects  were  impact, 
risk,  and  cost.  The  first  two  were  somewhat  similar  to  two  of  the  factors 
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Table  2.4-1,  Proposed  AST  Applications/Projects  j 


CREW  STATION  DESIGN 

User  Studies 
Vocabulary  Development 
Development  of  a Syntactic  Handler 
P’  )ed1zation  of  ASR  Equipment 
Empirical  Comparison  of  SENSO  1 & 2 Station 
Empirical  Comparison  of  Pilot  Station 
Empirical  Comparison  of  TACCO  Station 
Microphone  Use 

Speech  Recognition  Equipment  Multiplexing 
Voice  Generation  Personality  Development 
Automated  Voice  Data  Retrieval  System 
Flight  Instrument  Information  System 
Checklist  Prompts  - Procedures  Monitoring 
AST  Design  Criteria  Development 


PERFORMANCE  MEASUREMENT 

Aircraft  Evaluation  Aids 

Workoad  Monitoring 

Stress  Monitoring  Feasibility 

Procedures  Monitoring 

Workload  Optimized  Display  Systems 


TRAINING 

Simulated  Crew  Member 
Replacement  of  Device  Operator 
Demonstration  of  the  Transfer  of  Training 
Automated  Cockpit  Familiarity  Training 


RESEARCH  AND  DEVELOPMENT 

Continuous  Speech  Recognition 
Speaker  Independence 
User  Acceptance  Studies 
Feature  Extractor  Control 
Voice  Recognition  Feedback 

MAINTENANCE 

Cockpit  Maintenance  Support 
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used  to  rate  the  crew  station  design  tasks  (Reference  Section  2,2.4). 


a.  The  impact  characteristic  includes  the  factor  of  practicality 
or  probable  utility.  It  was  rated  on  the  basis  of  estimated 
advantage  to  future  users.  The  projects  receiving  the  best 
ratings  (lowest  numerical  value)  in  this  characteristic  cate- 
gory are  the  ones  that  are  the  easiest  to  justify  in  the  pro- 
gram plan. 

b.  Risk,  or  risk  of  project  success,  includes  such  factors  as  ASR 
required  time  delay,  accuracy  rates,  and  maximum  background  noise 
allowable.  Unassigned  variables  such  as  availability  of  resources, 
use  of  known  methods,  user  acceptance,  probable  system  interface, 
probable  mission  duration/ fatigue,  and  required  crew  intercom  use 
are  also  included  in  the  risk  evaluation  characteristic. 

c.  The  cost  characteristic  is  based  upon  estimates  as  to  the 
probable  availability  of  facilities,  equipment,  personnel, 
development  costs  and  time  required  to  evaluate  the  activities/ 
projects. 

2.4.2  Trade-Off  Matrix 

Each  of  the  activities/projects  has  been  listed  in  the  Table  2.4-2  matrix 
along  with  the  three  evaluation  characteristics.  Each  activity/project 
was  evaluated  with  a five  point  rating  system  on  the  basis  of  how  well  it 
met  each  evaluation  criteria  when  compared  to  the  other  projects.  The  pro- 
jects that  were  evaluated  as  having  the  greatest  impact,  most  probability 
of  success,  and/or  least  cost,  received  the  best  ratings  (score  or  value 
of  1).  Conversely,  the  projects  which  seemed  to  have  the  least  utility, 
success,  and/or  most  cost,  received  the  worst  ratings  (fives).  Each  pro- 
ject was  rated  in  comparison  to  each  characteristic  by  three  project  per- 
sons and  the  assigned  ratings  were  then  totaled. 

It  should  be  noted  that  no  evaluation  has  been  assigned  to  the  factor  of 
present  feasibility.  It  was  felt  that  this  characteristic  should  be  dealt 
with  by  scheduling  and  recommended  assignment  of  research  and  development 
funds  according  to  the  program  acquisition  cycle  categories  of  6,1,  6.2,  and 
6.3. 
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Table  2.4-2,  Trade-off  Matrix 


Project 

Impact 

Factor 

IM1+I2.I3 

Risk 

Factor 

0=11, 

Cost 

Factor 

C=C^+C2+C3 

Total 

Evaluation 

T=3I+R+C 

User  Voice  Studies 

7 

5 

6 

32 

Vocabulary  Development 

8 

3 

3 

30 

Devel . of  a Syntactic  Handler 

7 

5 

3 

29 

Ruggedization  of  ASR  Equip. 

5 

4 

9 

28 

Empirical  SENSO  1-2  Sta. 

Comp. 

5 

9 

12 

36 

Empirical  Pilot  Sta.  Comp. 

6 

9 

12 

39 

Empirical  TACCO  Sta.  Comp. 

6 

11 

12 

41 

Throat  Microphone  Use 

7 

5 

4 

30 

ASR  Equip.  Multiplexing 

11 

6 

7 

46 

Voice  Gen.  Personality  Devel. 

11 

5 

4 

42 

Voice  Data  Retrieval 

7 

5 

6 

32 

Flight  Instr.  Info.  Sys. 

5 

7 

8 

30 

Checklist  Prompts-Procedures 

4 

6 

10 

28 

AST  Design  Crit.  Dev. 

8 

6 

3 

33 

Aircraft  Eval . Aids 

11 

8 

9 

50 

Workload  Monitoring 

9 

9 

10 

46 

Stress  Monitoring  Feasibility 

12 

5 

3 

44 

Procedures  Monitoring 

8 

9 

7 

40 

Workload  Optimized  Display 

11 

12 

9 

54 

Simulated  Crew  Member 

4 

7 

9 

28 

Replace  Trn.  Device  Oper. 

8 

6 

6 

36 

Training  Transfer  Demo. 

10 

9 

5 

44 

Auto  Cockpit  Fam  Training 

8 

6 

9 

39 

Continuous  Speech  Recog. 

3 

12 

15 

36 

Speaker  Independence 

4 

10 

6 

28 

User  Acceptance  Studies 

6 

5 

3 

26 

Feature  Extractor  Control 

13 

12 

9 

60 

Cockpit  Maintenance  Support 

11 

7 

6 

46 

Voice  Recognition  Feedback 

9 

6 

5 

38 

The  following  describes  the  factor  rating  values. 

a.  IMPACT  (I^) 

Range  - 1 - Best  or  most  potential  benefit 
5 - Worst  or  least  potential  benefit 

b.  RISK  (R^) 

Range  - 1 - Best  or  least  chance  of  failure 
5 - Worst  or  most  chance  of  failure 

c.  COST  (C^)  Total  project  expenditures  using  1978  dollars. 

Range  - 1 - 50K  to  lOOK 

2 - lOOK  to  250K 

3 - 250K  to  BOOK 

4 - BOOK  to  lOOOK 

5 - lOOOK  to  5000K 

d.  TOTAL  EVALUATION  FACTOR  (T)  A factor  used  to  rank  the  various 

projects  in  terms  of  relative  desirability.  Impact 
has  been  weighted  by  a factor  of  3 to  emphasize  its 
importance  over  cost  and  risk.  The  lower  the  value 
of  "T"  the  more  attractive  it  is  with  respect  to  its 
implementation. 

Range  - T = 26  - Best 
T = 54  - Worst 

Because  three  project  persons  contributed  ratings  to  each  evaluation  factor, 
the  individual  ratings  were  summed  for  each  factor.  Then  a total  evaluation 
value,  T,  was  calculated  by  simply  adding  three  times  the  total  impact  factor 
plus  the  total  risk  and  cost  factors.  The  total  evaluation  value  (T)  was  used 
to  rank  the  29  projects.  Using  a subjective  criterion  of  50  T points,  three 
projects  were  deleted,  reducing  the  total  recommended  projects  to  26. 

The  Prioritized  Project  List  is  shown  in  Table  2.4-3  and  shows  the  projects 
ranked  in  order  of  their  Total  Evaluation  Factor  (T).  Also  included  in  this 
table  is  the  potential  sponsoring/monitoring  agency,  the  cost  factor  (C)  where 
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Table  2.4-3,  Prioritized  Project  List 


Number 

Total  Eval . 
Score 

Project 

Agency 

Cost 

Factor 

Duration 

(Months) 

1 

26 

User  Acceptance  Studies 

ONR 

3 

4 

2 

28 

Ruggedization  of  ASR  Equip. 

NAOC 

9 

12 

3 

28 

Checklist  Prompts-Procedures 

NADC 

10 

24 

4 

28 

Speaker  Independence 

ONR/NTEC 

6 

12 

5 

28 

Simulated  Crew  Member 

NTEC 

9 

30 

6 

29 

Devel.  of  a Syntactic  Handler 

NADC 

3 

12 

7 

30 

Vocabulary  Development 

ONR/ NADC 

3 

8 

8 

30 

Microphone  Use 

NADC 

4 

3 

9 

30 

Flight  Instr.  Info.  Sys. 

NADC 

8 

24 

10 

32 

User  Voice  Studies 

ONR 

6 

24 

n 

32 

Voice  Data  Retrieval 

NADC 

6 

18 

12 

33 

AST  Design  Criteria  Dev. 

NADC 

3 

12 

13 

36 

Continuous  Speech  Recog. 

ONR 

15 

14 

36 

Replace  Trn.  Device  Oper. 

NTEC 

6 

12 

15 

36 

Empirical  SENSO  1-2  Sta.  Comp. 

NADC 

12 

42 

16 

38 

Voice  Recognition  Feedback 

ONR/ NADC 

5 

12 

17 

39 

Auto  Cockpit  Fam  Training 

NTEC 

9 

24 

18 

39 

Empirical  Pilot  Sta.  Comp. 

NADC 

12 

36 

19 

40 

Procedures  Monitoring 

NADC 

7 

24 

20 

41 

Empirical  TACCO  Sta.  Comp. 

NADC 

12 

36 

21 

42 

Voice  Gen.  Personality  Devel. 

ONR 

4 

6 

22 

44 

Stress  Monitoring  Feasibility 

ONR/ NADC 

3 

6 

23 

44 

Training  Transfer  Demo. 

NTEC 

5 

12 

24 

46 

Cockpit  Maintenance  Support 

NADC/NTEC 

6 

12 

25 

46 

Workload  Monitoring 

NADC 

10 

24 

26 

46 

ASR  Equipment  Multiplexing 

ONR/ NADC 

7 

24 

I 

i 

I 
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C is  the  total  of  three  ratings  by  separate  evaluators,  and  the  estimated 
project  duration  in  months.  Note  that  the  cost  factor  (C)  must  be  divided 
by  three  if  it  is  to  be  used  to  correlate  to  the  actual  dollar  values  listed 
in  the  cost  description. 


3.0  CONaUSIONS 


3.1  General  Conclusions  and  Recommendations 
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The  specific  conclusions/recommendations  of  this  report  are  those  projects  listed 
in  Table  2.4-3  and  detailed  in  the  Program  Plan.  The  general  conclusions  and 
reconmendatlons  are  as  indicated  in  the  following  sections. 

3.1.1  Conclusions 

a.  AST  can  be  beneficially  applied  to  a variety  of  Navy  operational  and  training 

programs  including: 

/ 

1.  New  generation  patrol  aircraft  design. 

2.  Simulators  and  crew  procedures  trainers 

3.  Operational  ground  support  including  maintenance 

b.  AST  application  should  proceed  at  a rate  warranted  by  positive  empirical 
results  (from  proposed  programs).  Application  should  be  tailored  to  use  the 
advanteages  of  AST. 

c.  AST  has  better  applicability  for  some  crew  positions  than  others. 

d.  The  end  result  of  the  6 year  plan  will  be; 

1.  A developed  technology  ready  for  application 

2.  Guidelines  for  its  application 

3.  Immediate  improvements  in  training  and  performance  measurement 

3.1.2  Recommendations 

a.  Implement  the  Plan  as  closely  as  possible. 

b.  Initiate  a follow-up  (sequel)  to  this  planning  program  to  investigate  the 
progress  of  all  projects  in  the  plan  and  revise  plans  or  projects  as  necessary  - 
Budget  $100K. 

c.  Effect  positive  coordination  between  all  Navy  organizations  involved  so  as  to 
produce  the  best  overall  result  for  the  Navy  (Fleet  Aviation). 

3.2  Schedule 

As  a part  of  the  process  of  developing  the  details  for  accomplishment  of  the 
applications/projects  a task  schedule  breakdown  was  constructed  for  each.  This 
analysis  helped  determine  the  total  time  required  to  perform  each  project.  How- 
ever, much  of  the  purpose  of  this  study  contract  was  to  assist  Navy  planners  in 
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\ determining  just  when  to  perform  which  tasks.  This  determination  was  made 

after  the  projects  were  selected  and  prioritized.  The  overall  scheduling  was 
based  on  the  following  four  factors: 

a.  Certain  projects  required  other  projects  to  precede  them  and  must  therefore 
occur  later  in  the  overall  schedule. 

I b.  The  estimated  length  of  some  projects  required  that  they  be  started  relatively 

I early  if  they  were  to  be  completed  during  the  six  year  total  time  period. 

I 

i c.  Although  not  significant,  the  AST  state-of-the-art  required  for  some  pro- 

jects indicated  that  they  could  be  better  accomplished  at  a later  date. 

d.  In  order  to  equalize  the  budget  loading  over  the  six  year  period  some  tasks 
were  shifted  to  the  right  on  the  schedule  (delayed). 

i 

Consideration  of  the  project  sequencing  requirements  alone,  i.e.,  which  projects 
were  dependent  on  the  accomplishment  of  which  other  projects,  lead  to  the 
development  of  the  Figure  3.2-1  Suggested  Project  Sequencing  Chart.  This  chart 
is  somewhat  similar  to  a system  engineering  PERT  chart.  The  projects  on  the 
left  should  be  accomplished  before  those  on  the  right  side.  The  arrows  indicate 
the  general  order  of  accomplishment. 

Figure  3.2-2  is  the  Integrated  AST  Projects  Schedule.  This  schedule  takes  the 
data  from  Figure  3.2-1  and  adds  the  factors  of  project  length,  AST  state-of-the- 
art,  and  budget  leveling  over  a six  year  period  from  October  1978  to  October  1984. 
The  fiscal  years  are  used  for  budgeting  compatibility. 
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Figure  2.2  2:  Integrated  AST  Projects  Schedule 
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